Skip to content

feat: use positions based on a forest with 63 rows#104

Merged
Davidson-Souza merged 1 commit intomit-dci:mainfrom
Davidson-Souza:feature/translate
Mar 17, 2026
Merged

feat: use positions based on a forest with 63 rows#104
Davidson-Souza merged 1 commit intomit-dci:mainfrom
Davidson-Souza:feature/translate

Conversation

@Davidson-Souza
Copy link
Copy Markdown
Collaborator

@Davidson-Souza Davidson-Souza commented Mar 11, 2026

When we compute the position of a node, for any node not in the 0th row, their position depends on how many leaves there are. This happens because the 0th row's size is allocated to the nearest power of two that can fit that many leaves. Therefore, in a forest with 6 leaves, the bottom row goes from zero through 7, the row 1 from 8 through 11 (the size of each row halves as you move up). If you add three extra UTXOs, growing the forest to nine leaves, adding the 9th will require allocating 16 0-row leaves, row 1 therefore goes from 16 to 23 and so on.

If leaves always stay at the bottom, that fine. Nothing at the bottom ever needs to care about this, because there's no row before it to grow and shift
their positions. However, leaves do move up during deletions. For that
reason, whenever the forest grow, all targets that aren't at the bottom needs to be updated.

Now imagine that we want to keep a leaf map that maps leaf_hash -> position within the forest: this works fine, we know where a node must go when deleting, by calling [parent] with their current position and num_leaves. But now imagine the forest has to grow: we need to go through the map and update all non-row 0 leaves. This could potentially involve going through millions of UTXOs and update one-by-one. Note that we can find the next position, it's not super efficient but works (see [crate::proof::Proof::maybe_remap] for more details), but doing this for every UTXO that isn't at the bottom is too expensive, even though it happens exponentially less frequently, when it happens, it's going to take an absurd amount of time and potentially stall the Utreexo network for hours.

For that reason, we communicate positions as if the forest is always filled with the maximum amount of rows we can possibly have, which is 63. Therefore, those positions never need to be remapped. Internally, we still use the dynamic size, and use this function to translate between the two.

@Davidson-Souza Davidson-Souza changed the title feat: use positions based on a forest with 63 leaves feat: use positions based on a forest with 63 rows Mar 11, 2026
Cargo.toml Outdated
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
bitcoin_hashes = { version = "0.19", default-features = false }
bitcoin_hashes = { version = "0.20.0", default-features = false }
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
bitcoin_hashes = { version = "0.20.0", default-features = false }
bitcoin_hashes = { version = "0.20", default-features = false }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

102 already fixes this

src/lib.rs Outdated
/// dividing this by 90,000 we get 102,481,911,520,608 blocks
/// it would take 3,249,680 years to mine that many blocks...
///
/// For the poor soul in 3,249,682 who need to fix this hard-fork, here's what you gotta do:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lmao what is this documentation

@Davidson-Souza Davidson-Souza force-pushed the feature/translate branch 2 times, most recently from f9d7b84 to 854f4f3 Compare March 11, 2026 18:43
@Davidson-Souza
Copy link
Copy Markdown
Collaborator Author

rebased after #102

@JoseSK999
Copy link
Copy Markdown
Contributor

For that reason, we communicate positions as if the forest is always filled with the maximum amount of leaves we can possibly have, which is 63.

You mean, 2^63.

Increasing the number would only require a local remap and serialization change? All historic proofs and roots stay the same?

@Davidson-Souza
Copy link
Copy Markdown
Collaborator Author

You mean, 2^63.

I meant to say rows, not leaves. 63 rows = 2ˆ63 leaves

Increasing the number would only require a local remap and serialization change? All historic proofs and roots stay the same?

You don't need to change it on-disk, but you do need to translate before sending it to someone.

Copy link
Copy Markdown
Contributor

@JoseSK999 JoseSK999 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doc nits

src/lib.rs Outdated
/// dividing this by 90,000 we get 102,481,911,520,608 blocks
/// it would take 3,249,680 years to mine that many blocks...
///
/// For the poor soul in 3,249,682 A.D, who need to fix this hard-fork, here's what you gotta do:
Copy link
Copy Markdown
Contributor

@JoseSK999 JoseSK999 Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You meant adding the 2,000 years that have already passed, but this is just a pessimistic estimate anyway

Suggested change
/// For the poor soul in 3,249,682 A.D, who need to fix this hard-fork, here's what you gotta do:
/// For the poor soul in 3,251,680 A.D., who need to fix this hard-fork, here's what you gotta do:

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the punch gets even funnier if I add the two there (this is obviously an Easter egg and not meant to be taken seriously).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The guy reading this in 3,251,680 will actually take this seriously

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3,251,682*, in 3,251,680 they will be fighting whether the bug is real

src/util/mod.rs Outdated
Comment on lines +32 to +64
/// Translates targets from a forest with `from_rows` to a forest with `to_rows`.
///
/// When we compute the position of a node, for any node not in the 0th row, their position depends
/// on how many leaves there are. This happens because the 0th row's size is allocated to the
/// nearest power of two that can fit that many leaves. Therefore, in a forest with 6 leaves, the
/// bottom row goes from zero through 7, the row 1 from 8 through 11 (the size of each row
/// halves as you move up). If you add three extra UTXOs, growing the forest to nine leaves, adding
/// the 9th will require allocating 16 0-row leaves, row 1 therefore goes from 16 to 23 and so on.
///
/// If leaves always stay at the bottom, that fine. Nothing at the bottom ever needs to care about
/// this, because there's no row before it to grow and shift their positions. However, leaves
/// **do** move up during deletions. For that reason, whenever the forest grow, all targets that
/// aren't at the bottom needs to be updated.
///
/// Now imagine that we want to keep a leaf map that maps leaf_hash -> position within the forest:
/// this works fine, we know where a node must go when deleting, by calling [`parent`] with their
/// current position and `num_leaves`. But now imagine the forest has to grow: we need to go through
/// the map and update all non-row 0 leaves. This could potentially involve going through millions
/// of UTXOs and update one-by-one. Note that we can find the next position, it's not super
/// efficient but works (see [`crate::proof::Proof::maybe_remap`] for more details), but doing this
/// for every UTXO that isn't at the bottom is too expensive, even though it happens exponentially
/// less frequently, when it happens, it's going to take an absurd amount of time and potentially
/// stall the Utreexo network for hours.
///
/// For that reason, we communicate positions as if the forest is always filled with the maximum
/// amount of leaves we can possibly have, which is 63. Therefore, those positions never need to be
/// remapped. Internally, we still use the dynamic size, and use this function to translate between
/// the two.
///
/// # Implementation
///
/// This function simply computes how far away from the start of the row this leaf is, then use it
/// to offset the same amount in the new structure.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: small text style modification

Suggested change
/// Translates targets from a forest with `from_rows` to a forest with `to_rows`.
///
/// When we compute the position of a node, for any node not in the 0th row, their position depends
/// on how many leaves there are. This happens because the 0th row's size is allocated to the
/// nearest power of two that can fit that many leaves. Therefore, in a forest with 6 leaves, the
/// bottom row goes from zero through 7, the row 1 from 8 through 11 (the size of each row
/// halves as you move up). If you add three extra UTXOs, growing the forest to nine leaves, adding
/// the 9th will require allocating 16 0-row leaves, row 1 therefore goes from 16 to 23 and so on.
///
/// If leaves always stay at the bottom, that fine. Nothing at the bottom ever needs to care about
/// this, because there's no row before it to grow and shift their positions. However, leaves
/// **do** move up during deletions. For that reason, whenever the forest grow, all targets that
/// aren't at the bottom needs to be updated.
///
/// Now imagine that we want to keep a leaf map that maps leaf_hash -> position within the forest:
/// this works fine, we know where a node must go when deleting, by calling [`parent`] with their
/// current position and `num_leaves`. But now imagine the forest has to grow: we need to go through
/// the map and update all non-row 0 leaves. This could potentially involve going through millions
/// of UTXOs and update one-by-one. Note that we can find the next position, it's not super
/// efficient but works (see [`crate::proof::Proof::maybe_remap`] for more details), but doing this
/// for every UTXO that isn't at the bottom is too expensive, even though it happens exponentially
/// less frequently, when it happens, it's going to take an absurd amount of time and potentially
/// stall the Utreexo network for hours.
///
/// For that reason, we communicate positions as if the forest is always filled with the maximum
/// amount of leaves we can possibly have, which is 63. Therefore, those positions never need to be
/// remapped. Internally, we still use the dynamic size, and use this function to translate between
/// the two.
///
/// # Implementation
///
/// This function simply computes how far away from the start of the row this leaf is, then use it
/// to offset the same amount in the new structure.
/// Translates targets from a forest with `from_rows` to a forest with `to_rows`.
///
/// When we compute the position of a node, any node not in row 0 has a position that depends
/// on how many leaves there are. This happens because row 0 is allocated to the nearest power of
/// two that can fit that many leaves. Therefore, in a forest with 6 leaves, the bottom row goes
/// from 0 through 7, and row 1 goes from 8 through 11 (the size of each row halves as you move
/// up). If you add three extra UTXOs, growing the forest to 9 leaves, adding the 9th will require
/// allocating 16 row-0 leaves; row 1 therefore goes from 16 through 23, and so on.
///
/// If leaves always stayed at the bottom, that's fine. Nothing at the bottom ever needs to care
/// about this, because there is no row below it whose growth would shift its position. However,
/// leaves **do** move up during deletions. For that reason, whenever the forest grows, all targets
/// that are not at the bottom need to be updated.
///
/// Now imagine that we want to keep a leaf map from `leaf_hash` to position within the forest:
/// this works fine, and we know where a node must go when deleting by calling [`parent`] with its
/// current position and `num_leaves`. But now imagine the forest has to grow: we need to go
/// through the map and update all non-row-0 leaves. This could potentially involve going through
/// millions of UTXOs and updating them one by one. Note that we can find the next position; it is
/// not super efficient, but it works (see [`crate::proof::Proof::maybe_remap`] for more details).
/// But doing this for every UTXO that is not at the bottom is too expensive. Even though it
/// happens exponentially less frequently, when it does happen, it is going to take an absurd
/// amount of time and could potentially stall the Utreexo network for hours.
///
/// For that reason, we communicate positions as if the forest was always filled with the maximum
/// number of leaves we can possibly have, which is 63. Therefore, those positions never need to be
/// remapped. Internally, we still use the dynamic size, and use this function to translate between
/// the two.
///
/// # Implementation
///
/// This function simply computes how far from the start of the row this leaf is, then uses that
/// offset in the new structure.

src/lib.rs Outdated
///
/// # Calculations
///
/// If you think: "but... is 63 enough space"? Well... Assuming there's around 999,000 WU
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// If you think: "but... is 63 enough space"? Well... Assuming there's around 999,000 WU
/// If you think: "but... is 63 enough space"? Well... assuming there's around 999,000 WUs

src/lib.rs Outdated
/// If you think: "but... is 63 enough space"? Well... Assuming there's around 999,000 WU
/// available on each block (let's account for header and coinbase), a non-segwit transaction's
/// size is:
/// 4 (version) + 1 (vin count) + 41 (input) + 5 (vout for a large number of inputs) + 10N + 4
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean a large number of outputs?

src/lib.rs Outdated
Comment on lines +50 to +51
/// - Change the leaf_data type to a u128 or something q128 if Quantum Bits are the fashionable
/// standard
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// - Change the leaf_data type to a u128 or something q128 if Quantum Bits are the fashionable
/// standard
/// - Change the `leaf_data` type to u128, or q128 if Quantum Bits are the fashionable standard

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noo you just made this one liner without the changeee @Davidson-Souza (anyway you need to re-push to fix MSRV)

@Davidson-Souza
Copy link
Copy Markdown
Collaborator Author

Pushed 95384ad applying docs improvements suggested by @JoseSK999

Copy link
Copy Markdown
Contributor

@JoseSK999 JoseSK999 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for this 😢

src/util/mod.rs Outdated

/// Translates targets from a forest with `from_rows` to a forest with `to_rows`.
///
/// When we compute the position of a node, for any node not in row 0 has a position that depends
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing from my prev comment

Suggested change
/// When we compute the position of a node, for any node not in row 0 has a position that depends
/// When we compute the position of a node, any node not in row 0 has a position that depends

src/util/mod.rs Outdated
Comment on lines +37 to +39
/// from 0 through 7, and row 1 goes from 8 through 11 (the size of each row halves as you
/// move up). If you add three extra UTXOs, growing the forest to 9 leaves, adding the 9th
/// will require allocating 16 row-0 leaves; row 1 therefore goes from 16 though 23, and so on.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also missing

Suggested change
/// from 0 through 7, and row 1 goes from 8 through 11 (the size of each row halves as you
/// move up). If you add three extra UTXOs, growing the forest to 9 leaves, adding the 9th
/// will require allocating 16 row-0 leaves; row 1 therefore goes from 16 though 23, and so on.
/// from 0 through 7, and row 1 goes from 8 through 11 (the size of each row halves as you move
/// up). If you add three extra UTXOs, growing the forest to 9 leaves, adding the 9th will require
/// allocating 16 row-0 leaves; row 1 therefore goes from 16 through 23, and so on.

src/util/mod.rs Outdated
Comment on lines +41 to +44
/// If leaves always stayed at the bottom, that fine. Nothing at the bottom ever needs to care about
/// this, because there is no row below it whose growth would shift its positions. However, leaves
/// **do** move up during deletions. For that reason, whenever the forest grows, all targets that
/// are not at the bottom needs to be updated.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// If leaves always stayed at the bottom, that fine. Nothing at the bottom ever needs to care about
/// this, because there is no row below it whose growth would shift its positions. However, leaves
/// **do** move up during deletions. For that reason, whenever the forest grows, all targets that
/// are not at the bottom needs to be updated.
/// If leaves always stayed at the bottom, that's fine. Nothing at the bottom ever needs to care
/// about this, because there is no row below it whose growth would shift its position. However,
/// leaves **do** move up during deletions. For that reason, whenever the forest grows, all targets
/// that are not at the bottom need to be updated.

src/util/mod.rs Outdated
Comment on lines +46 to +54
/// Now, imagine that we want to keep a leaf map from `leaf_hash` to position within the forest:
/// this works fine, and we know where a node must go when deleting, by calling [`parent`] with their
/// current position and `num_leaves`. But now imagine the forest has to grow: we need to go through
/// the map and update all non-row-0 leaves. This could potentially involve going through millions
/// of UTXOs and update them one by one. Note that we can find the next position, it is not super
/// efficient, but it works (see [`crate::proof::Proof::maybe_remap`] for more details). But doing this
/// for every UTXO that are not at the bottom is too expensive. Even though it happens exponentially
/// less frequently, when it happens, it is going to take an absurd amount of time and potentially
/// stall the Utreexo network for hours.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// Now, imagine that we want to keep a leaf map from `leaf_hash` to position within the forest:
/// this works fine, and we know where a node must go when deleting, by calling [`parent`] with their
/// current position and `num_leaves`. But now imagine the forest has to grow: we need to go through
/// the map and update all non-row-0 leaves. This could potentially involve going through millions
/// of UTXOs and update them one by one. Note that we can find the next position, it is not super
/// efficient, but it works (see [`crate::proof::Proof::maybe_remap`] for more details). But doing this
/// for every UTXO that are not at the bottom is too expensive. Even though it happens exponentially
/// less frequently, when it happens, it is going to take an absurd amount of time and potentially
/// stall the Utreexo network for hours.
/// Now imagine that we want to keep a leaf map from `leaf_hash` to position within the forest:
/// this works fine, and we know where a node must go when deleting by calling [`parent`] with its
/// current position and `num_leaves`. But now imagine the forest has to grow: we need to go
/// through the map and update all non-row-0 leaves. This could potentially involve going through
/// millions of UTXOs and updating them one by one. Note that we can find the next position; it is
/// not super efficient, but it works (see [`crate::proof::Proof::maybe_remap`] for more details).
/// But doing this for every UTXO that is not at the bottom is too expensive. Even though it
/// happens exponentially less frequently, when it does happen, it is going to take an absurd
/// amount of time and could potentially stall the Utreexo network for hours.

src/lib.rs Outdated
Comment on lines +37 to +52
/// 4 (version) + 1 (vin count) + 41 (input) + 5 (vout for a large number of outputs) + 10N + 4
/// (locktime)
///
/// N is how many outputs we have (we are considering outputs with amount and a zero-sized
/// script), for 999,000 WU we can fit
/// 55 + 10N <= 999,000
/// N ~= 90k outputs (a little over)
///
/// 2^63 = 9,223,372,036,854,775,808
/// dividing this by 90,000 we get 102,481,911,520,608 blocks
/// it would take 3,249,680 years to mine that many blocks...
///
/// For the poor soul in 3,249,682 A.D., who need to fix this hard-fork, here's what you gotta do:
/// - Change the leaf_data type to a u128 or something q128 if Quantum Bits are the fashionable standard
/// - Change `MAX_FOREST_ROWS` to 128 or higher in `lib.rs`
/// - Modify [`start_position_at_row`] to avoid overflows.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// 4 (version) + 1 (vin count) + 41 (input) + 5 (vout for a large number of outputs) + 10N + 4
/// (locktime)
///
/// N is how many outputs we have (we are considering outputs with amount and a zero-sized
/// script), for 999,000 WU we can fit
/// 55 + 10N <= 999,000
/// N ~= 90k outputs (a little over)
///
/// 2^63 = 9,223,372,036,854,775,808
/// dividing this by 90,000 we get 102,481,911,520,608 blocks
/// it would take 3,249,680 years to mine that many blocks...
///
/// For the poor soul in 3,249,682 A.D., who need to fix this hard-fork, here's what you gotta do:
/// - Change the leaf_data type to a u128 or something q128 if Quantum Bits are the fashionable standard
/// - Change `MAX_FOREST_ROWS` to 128 or higher in `lib.rs`
/// - Modify [`start_position_at_row`] to avoid overflows.
/// `4 (version) + 1 (vin count) + 41 (input) + 5 (vout for many outputs) + 10N + 4 (locktime)`
///
/// `N` is how many outputs we have (we are considering outputs with amount and a zero-sized
/// script), for 999,000 WUs we can fit:
/// - `55 + 10N <= 999,000`
/// - `N ~= 90k` outputs (a little over)
///
/// Since `2^63 = 9,223,372,036,854,775,808`, if you divide this by 90,000 we get
/// 102,481,911,520,608 blocks. It would take us 3,249,680 years to mine that many blocks.
///
/// For the poor soul in 3,249,682 A.D., who needs to fix this hard-fork, here's what you gotta do:
/// - Change the `leaf_data` type to u128, or q128 if Quantum Bits are the fashionable standard.
/// - Change `MAX_FOREST_ROWS` to 128 or higher in `lib.rs`
/// - Modify [`start_position_at_row`] to avoid overflows.

@luisschwab
Copy link
Copy Markdown
Contributor

Should add a typo finder on CI

@Davidson-Souza
Copy link
Copy Markdown
Collaborator Author

Fixed the MSRV error and applied docs changes suggestions by @JoseSK999

src/lib.rs Outdated
Comment on lines +49 to +50
/// - Change `MAX_FOREST_ROWS` to 128 or higher in `lib.rs`
/// - Modify [`start_position_at_row`] to avoid overflows.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional nit, broken link

Suggested change
/// - Change `MAX_FOREST_ROWS` to 128 or higher in `lib.rs`
/// - Modify [`start_position_at_row`] to avoid overflows.
/// - Change `MAX_FOREST_ROWS` to 128 or higher in `lib.rs`.
/// - Modify [`util::start_position_at_row`] to avoid overflows.

src/proof/mod.rs Outdated
.targets
.iter()
.copied()
.map(|pos| translate(pos, 63, total_rows))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't u use the constant here

src/proof/mod.rs Outdated
.targets
.iter()
.copied()
.map(|pos| translate(pos, 63, total_rows))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and here

@Davidson-Souza
Copy link
Copy Markdown
Collaborator Author

Got this warning:
....

Hmm, utils is private (pub(crate)). I think we need to escape this since it won't render when building docs. Should we make it [utils::translate] or just [translate]?

@JoseSK999
Copy link
Copy Markdown
Contributor

JoseSK999 commented Mar 16, 2026

That was my mistake (I made it public), that's why I removed the comment 😂 only the review comments are relevant

@Davidson-Souza
Copy link
Copy Markdown
Collaborator Author

@JoseSK999 used the constants where applicable.

@luisschwab
Copy link
Copy Markdown
Contributor

I'll review this one later today.

@Davidson-Souza
Copy link
Copy Markdown
Collaborator Author

Rebased, since 2774072 is already on main


#[test]
fn test_start_position_at_row() {
assert_eq!(start_position_at_row(1, 12), 4096);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Last nit: I feel like we could add a few more test cases here, with different number of rows and row positions.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

When we compute the position of a node, for any node not in the 0th row,
their position depends on how many leaves there are. This happens because
the 0th row's size is allocated to the nearest power of two that can
fit that many leaves. Therefore, in a forest with 6 leaves, the bottom row
goes from zero through 7, the row 1 from 8 through 11 (the size of  each row
halves as you move up). If you add three extra UTXOs, growing the forest to
nine leaves, adding the 9th will require allocating 16 0-row leaves, row 1
therefore goes from 16 to 23 and so on.

If leaves always stay at the bottom, that fine. Nothing at the bottom ever
needs to care about this, because there's no row before it to grow and shift
 their positions. However, leaves **do** move up during deletions. For that
reason, whenever the forest grow, all targets that aren't at the bottom needs
to be updated.

Now imagine that we want to keep a leaf map that maps leaf_hash -> position
within the forest: this works fine, we know where a node must go when deleting,
by calling [`parent`] with their current position and `num_leaves`. But now
imagine the forest has to grow: we need to go through the map and update
all non-row 0 leaves. This could potentially involve going through millions of
UTXOs and update one-by-one. Note that we can find the next position, it's not super
efficient but works (see [`crate::proof::Proof::maybe_remap`] for more details),
but doing this for every UTXO that isn't at the bottom is too expensive, even
though it happens exponentially less frequently, when it happens, it's going to
take an absurd amount of time and potentially stall the Utreexo network for hours.

For that reason, we communicate positions as if the forest is always filled with
the maximum amount of leaves we can possibly have, which is 63. Therefore, those
positions never need to be remapped. Internally, we still use the dynamic size,
and use this function  to translate between the two.
Copy link
Copy Markdown
Contributor

@JoseSK999 JoseSK999 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 627664a

@JoseSK999
Copy link
Copy Markdown
Contributor

@luisschwab ur turn

Copy link
Copy Markdown
Contributor

@luisschwab luisschwab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 627664a

@luisschwab
Copy link
Copy Markdown
Contributor

for_respected_client

@Davidson-Souza Davidson-Souza merged commit dc95f50 into mit-dci:main Mar 17, 2026
14 checks passed
Davidson-Souza added a commit to getfloresta/Floresta that referenced this pull request Mar 17, 2026
3c3a939 chore: bump rustreexo (Davidson Souza)

Pull request description:

  ### Description and Notes

  ~~This PR updates rustreexo to `main` + mit-dci/rustreexo#104

  ~~I'll leave this as draft until we ship the next version, but this should help testing mit-dci/rustreexo#104

  This PR bumps rustreexo v0.4.0 -> v0.5.0

ACKs for top commit:
  moisesPompilio:
    ACK 3c3a939
  luisschwab:
    tACK 3c3a939
  JoseSK999:
    ACK 3c3a939; I had previously synced signet in this branch and currently it is still working for me (given I connect a good bridge)

Tree-SHA512: eb941c27b7d9de0e4a47e9f65912904d7cb6df3fc93fd114a412424140d94820eda58e1e2b71bb6ecc85a86612981589dc1269ca32743ac8fc1f4bf9247d3a74
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants